Papers with guardrail models
CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistants (2024.naacl-long)
Copied to clipboard
| Challenge: | A major challenge in deploying LLM-based virtual conversational assistants in real world settings is ensuring they operate within what is admissible for the task. |
| Approach: | They propose to use large language models (LLMs) to generate training data with two key LLM components: scenario-augmented generation and contrastive training examples. |
| Outcome: | The proposed model improves over baselines in multiple dialogue domains. |
FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing guardrail models for content moderation assume a fixed definition of harmfulness, but enforced strictness varies across platforms and evolves over time, resulting in brittle moderators. |
| Approach: | They propose a strictness-adaptive LLM moderation benchmark that enables controlled evaluation under multiple strictness regimes. |
| Outcome: | The proposed moderator performs better under one regime and under another, and is more robust under varying strictness. |
LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing model-based guardrails have not been designed for resource-constrained computational portable devices such as mobile phones. |
| Approach: | They propose a parameter-efficient guardrail adaptation method that relies on knowledge sharing between LLMs and guardrail models to adapt to content moderation tasks. |
| Outcome: | The proposed method outperforms existing guardrail methods with lower parameter overhead and higher accuracy on the generative task. |
On Guardrail Models’ Robustness to Mutations and Adversarial Attacks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative AI systems providing unsafe information has raised significant concerns, emphasizing the need for safety guardrails. |
| Approach: | They propose to evaluate 15 state-of-the-art guardrail models to assess their robustness to input mutations and adversarial attacks designed to bypass models’ safety alignment. |
| Outcome: | The proposed models are robust to input mutations and adversarial attacks that bypass models’ safety alignment. |
GuardBench: A Large-Scale Benchmark for Guardrail Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Lack of a standard benchmark for guardrail models poses significant evaluation issues . lack of standardized benchmark makes it hard to compare results across scientific publications. |
| Approach: | They propose a large-scale benchmark for guardrail models comprising 40 safety evaluation datasets. |
| Outcome: | The proposed model achieves competitive results without specific fine-tuning without the need for specific fine tuning. |
RST-Guarder: Enhancing Long-Context Robustness for Safeguards via RST Parsing and Probabilistic Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing guardrail models for harmful-content detection degrade on long-form inputs . Existing models are vulnerable to policy-violating responses, causing false positives based on benign content . |
| Approach: | They propose an inference-time method that improves harmful-content detection for long-form inputs without additional data curation or model training. |
| Outcome: | The proposed method improves harmful-content detection for long-form inputs without additional data curation or model training. |
Jailbreaking Large Language Models with Morality Attacks (2026.findings-acl)
Copied to clipboard
| Challenge: | Pluralism alignment is the goal of creating AI that can coexist with and serve morally multifaceted humanity. |
| Approach: | They propose to use jailbreak attacks to manipulate LLMs’ judgment over pluralistic values by using a morality dataset with 10.4K instances. |
| Outcome: | The proposed method exploits the persuasion abilities of LLMs to produce moral content over pluralistic values. |